Tagged robot sensor data

ABSTRACT

Methods, apparatus, systems, and computer-readable media are provided for creating, storing, and/or offloading tagged robot sensor data. In various implementations, a first plurality of sensor data points that are sampled by one or more sensors associated with a robot and that share a first attribute may be identified. Each of the first plurality of sensor data points may be tagged with a first tag, which may be indicative of the first attribute. A context in which a robot is operating may be identified. A first transport rule that governs how sensor data points tagged with the first tag are treated when the robot operates in the context may then be identified. At least a subset of the first plurality of tagged sensor data points may then be offloaded from the robot and/or stored locally on the robot pursuant to the first transport rule.

The present application is a continuation of U.S. patent applicationSer. No. 14/850,888, which was filed on Sep. 10, 2015 and issued as U.S.Pat. No. 10,118,296 on Nov. 6, 2018, and which is incorporated herein byreference.

BACKGROUND

Sensor data produced by a robot can be used for a variety of purposes,such as debugging the robot, gathering various statistics about how therobot operates, learning about an environment in which the robotoperates, etc. However, robots typically have numerous sensors thatproduce a large amount of sensor data. For example, many robots havemultiple operational components (e.g., joints, gears, drives, etc.),many of which include or act as sensors (e.g., torque sensors) thatcollect data points pertaining to the operational components. Manyrobots also include various standalone sensors, such as microphones,cameras, accelerometers, GPS sensors, proximity sensors, and so forth,each producing yet more sensor data. Depending on the robot's context,storing such large amounts of sensor data locally in memory of a robotmay not be practicable. It may be feasible to offload the sensor data toa remote computing system such as a logging server or debuggingworkstation under some circumstances, such as when availablecommunication bandwidth/reliability between the robot and the remotecomputing system is relatively high. However, robots, and particularlyautonomous or semi-autonomous robots, often operate in contexts in whichcommunication is limited.

SUMMARY

The present disclosure is generally directed to methods, apparatus, andcomputer-readable media (transitory and non-transitory) for creating,offloading, and/or storing tagged robot sensor data. Data points sampledby robot sensors may be “tagged” based on attributes of the data points,such as origin and/or data type. Then, tagged sensor data points may beautomatically stored in memory of the robot and/or offloaded to a remotecomputing system pursuant to one or more so-called “transport rules”that govern how tagged sensor data is offloaded and/or stored while therobot operates in various contexts. For example, if a robot is beingdebugged and therefore has a high reliability, high bandwidthcommunication channel with a debugging workstation, all or most taggedsensor data points may be offloaded to the workstation. However, if therobot is deployed in the field with only a weak wireless networkconnection, then all or a subset of sensor data tagged as relativelyhigh priority may be offloaded to a remote computing system, and sensordata tagged as relatively low priority may not be offloaded and/orinstead may be stored locally.

In some implementations, a method may be provided that includes:identifying, by a robot, a first plurality of sensor data points thatare sampled by one or more sensors associated with the robot and thatshare a first attribute; tagging, by the robot, each of the firstplurality of sensor data points with a first tag, wherein the first tagis indicative of the first attribute shared by the first plurality ofsensor data points; identifying, by the robot, a context in which therobot is operating; identifying, by the robot, a first transport rulethat governs how sensor data points tagged with the first tag aretreated when the robot operates in the context; and offloading orstoring, by the robot, at least a subset of the first plurality oftagged sensor data points pursuant to the first transport rule.

This method and other implementations of technology disclosed herein mayeach optionally include one or more of the following features.

In various implementations, the offloading may include transmitting atleast the subset of the first plurality of tagged sensor data points toa remote computing device in a manner selected based on the context inwhich the robot is operating. In various implementations, the subset maybe selected from the first plurality of sensor data points based on thecontext in which the robot is operating. In various implementations, thetransmitting may include transmitting at least the subset of the firstplurality of tagged sensor data points through one or more communicationchannels selected based on the context.

In various implementations, the context may include availablecommunication bandwidth between the robot and the remote computingdevice. In various implementations, the context may include reliabilityof a communication channel between the robot and the remote computingdevice. In various implementations, the context may include poweravailable to the robot. In various implementations, the context mayinclude available memory in the robot. In various implementations, thecontext may include an activity level of the robot.

In various implementations, the first attribute may include an origin ofthe first plurality of sensor data points. In various implementations,the first attribute may include a data type of the first plurality ofsensor data points. In various implementations, the method may furtherinclude: identifying, by the robot, a second plurality of sensor datapoints that are sampled by the one or more sensors associated with therobot and that share a second attribute; tagging, by the robot, each ofthe second plurality of sensor data points with a second tag, whereinthe second tag is different than the first tag and is indicative of thesecond attribute shared by the second plurality of sensor data points;identifying, by the robot, a second transport rule that governs how datapoints tagged with the second tag are treated when the robot operates inthe context; and offloading or storing, by the robot, at least a subsetof the second plurality of tagged sensor data points pursuant to thesecond transport rule.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method such as one or more of the methods described above. Yet anotherimplementation may include a system, such as a robot, that includesmemory and logic operable to execute instructions, stored in the memory,to implement one or more modules or engines that, alone or collectively,perform a method such as one or more of the methods described above.

It should be appreciated that all combinations of the foregoing conceptsand additional concepts described in greater detail herein arecontemplated as being part of the subject matter disclosed herein. Forexample, all combinations of claimed subject matter appearing at the endof this disclosure are contemplated as being part of the subject matterdisclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically depicts an example environment in which disclosedtechniques may be implemented, in accordance with variousimplementations.

FIG. 2 schematically depicts in more detail than FIG. 1 examplecomponents of a robot that may cooperatively tag sensor data anddistribute it pursuant to one or more transport rules, in accordancewith various implementations.

FIG. 3 depicts an example graphical user interface that may be operableto select and/or retrieve sensor data tagged with various tags, inaccordance with various implementations.

FIG. 4 depicts an example method for tagging and/or offloading/storingtagged sensor data, in accordance with various implementations.

FIG. 5 schematically depicts an example architecture of a computersystem.

DETAILED DESCRIPTION

FIG. 1 schematically depicts an example environment in which disclosedtechniques may be implemented to, in accordance with variousimplementations. A robot 100 may take various forms, including but notlimited to a telepresence robot, a robot arm, a humanoid, an animal, aninsect, an aquatic creature, a wheeled device, a submersible vehicle, anunmanned aerial vehicle (“UAV”), and so forth. Robot 100 may includelogic 102 operably coupled to one or more operational components 104,one or more end effectors 106, one or more standalone sensors 108,memory 110, one or more wired communication interfaces 112, and one ormore wireless communication interfaces 114 via one or more internalbuses 110. Logic 102 may, in various implementations, include one ormore processors, such as one or more so-called “real time processors”that are guaranteed to perform various operations within varioustemporal constraints.

As used herein, “operational components” 104 of robot 100 may refer toactuators, motors (e.g., servo motors), joints, shafts, gear trains,wheels, tracks, pumps (e.g., air or liquid), pistons, drives, or othercomponents that may create and/or undergo propulsion, rotation, and/ormotion to move components of robot 100 relative to each other, and/or tomove robot 100 as a whole. Some operational components may beindependently controllable, although this is not required. In someinstances, the more operational components robot 100 has, the moredegrees of freedom of movement it may have. As noted in the background,many operational components 104 include and/or are coupled with sensorsconfigured to provide raw data associated with operation of theoperational component 104. For example, a joint may include a torquesensor.

As used herein, “end effector” 106 may refer to a variety of tools thatmay be operated by robot 100 in order to accomplish various tasks. Forexample, some robots may be equipped with various types of “grippers,”including but not limited to “impactive” (e.g., “grab” objects usingopposing digits), “ingressive” (e.g., physically penetrating an objectusing pins, needles, etc.), “astrictive” (e.g., using suction or vacuumto pick up an object), or “contigutive” (e.g., using surface tension,freezing or adhesive to pick up object). More generally, other types ofend effectors may include but are not limited to drills, brushes,force-torque sensors, cutting tools, deburring tools, welding torches,and so forth.

Standalone sensors 108 may take various forms, including but not limitedto cameras, light sensors (e.g., passive infrared), pressure sensors,pressure wave sensors (e.g., microphones), proximity sensors, torquesensors, force sensors, radars, range finders, accelerometers,gyroscopes, compasses, position coordinate sensors (e.g., globalpositioning system, or “GPS”), speedometers, drop off sensors (e.g., todetect an edge of a raised surface), and so forth. While standalonesensors 108 are depicted as being integral with robot 100, this is notmeant to be limiting. In some implementations, standalone sensors 108may be located external to, but may be in direct or indirectcommunication with, robot 100. The term “sensor” as used herein withoutany modifiers (e.g., “standalone”) may refer to both operationalcomponents 104 and standalone sensors 108.

Memory 110 may come in various forms, including but not limited torandom access memory (“RAM”), read only memory (“ROM”), dynamic RAM(“DRAM”), flash, etc. In various implementations, logic 102 may executeone or more instructions contained in memory 110 to operate a sensordata management engine (“S.D.M.E.” in FIG. 1) 130. Sensor datamanagement engine 130 will be described in more detail with regard toFIG. 2.

Wired communication interface(s) 112 may come in various forms,including but not limited to Ethernet, USB, serial, PCIe external, etc.Wired communication interface 112 may be used to exchange data with oneor more remote computing systems through one or more wired communicationchannels 118. Similarly, wireless communication interface(s) 114 maycome in various forms that utilize various wireless communicationtechnologies, including but not limited to Wi-Fi, Bluetooth, cellular(e.g., 3G, 4G LTE, etc.), radio, and so forth. Wireless communicationinterface 114 may be operably coupled with one or more antenna 116, ofwhich two examples, 116A and 116B, are depicted in FIG. 1. Each antenna116 may be configured to transmit data provided (e.g., modulated) bywireless communication interface 114 over one or more wirelesscommunication channels 120A-B. As one non-limiting example, firstantenna 116A could be a Wi-Fi antenna, and second antenna 116B could bea Bluetooth antenna. While not required, it is often the case that wiredcommunication channels such as 118 are more reliable and/or have morebandwidth than wireless communication channels such as 120A-B.

Robot 100 may be in communication with various remote computing systemsfor various purposes, such as controlling robot 100 to various degrees,debugging robot 100, and so forth. For example, robot 100 may, in somecontexts such as when the robot is being programmed and/or debugged, bein communication with a debugging workstation 150 over wiredcommunication channel 118. Debugging workstation 150 may include one orcomputing systems connected by one or more networks (not depicted). Anexample of such a computing system is depicted schematically in FIG. 5.Various modules or engines may be implemented as part of debuggingworkstation 150 as software, hardware, or any combination of the two.For example, in FIG. 1, debugging workstation 150 includes a userinterface engine 152 and a logging engine 154.

User interface engine 152 may be configured to receive, as input,commands from various sources, such as human technicians. In someimplementations, user interface engine 152 may provide one or more userinterfaces, locally and/or distributed to remote computing devices(e.g., as interactive web pages), which may be operated by users such astechnicians to perform various tasks, such as programming and/ordebugging robot 100. In some implementations, user interface engine 152may provide an interface that is operable by a user to select howvarious types of sensor data sampled by standalone sensors 108 and/oroperational components 104 of robot 100 should be tagged. For example, auser may operate the interface to cause robot 100 to tag sensor datasampled by one sensor with a first tag, and sensor data sampled byanother sensor with a second tag (origin-based tagging). Additionally oralternatively, the user may operate the interface to configure robot 100to tag sensor data sampled while robot 100 operates in a first context(e.g., normal operation) with a first tag, to tag sensor data sampledwhile robot 100 operates in a second context (e.g., recalibration mode)with a second tag, to tag sensor data sample while robot 100 operates ina third context (e.g., error and/or collision response mode) with athird tag, and so on.

User interface engine 152 may provide the same or different graphicaluser interface (see FIG. 3) that is operable to configure how robot 100affirmatively offloads (e.g., pushes) tagged sensor data to a remotecomputing device (e.g., 150, 160), and/or how tagged sensor data isretrieved (e.g., pulled) from robot 100. For example, a user may wish toonly see a particular type of sensor data (i.e. tagged with a particulardata type tag) and/or sensor data sampled by a particular sensor (i.e.tagged with particular origin tag), such as data sampled by a particularjoint while robot 100 is in an error response mode. The user may, ineffect, define a transport rule “on the fly” that causes the robot tooffload only sensor data tagged with the targeted data type tag andtargeted origin tag. As this example makes clear, in someimplementations, sensor data points may be tagged with multiple tags.

Logging engine 154 may be configured to receive, e.g., from robot 100,sensor data that robot 100 offloads pursuant to one or more transportrules. Logging engine 154 may store such sensor data in an index 156. Inmany scenarios, records of sensor data in index 156 may be provided bylogging engine 154 for use in debugging robot 100. For example, a useroperating an interface (see FIG. 3) provided by user interface engine152 may request a particular type of sensor data, or sensor data sampledby a particular operational component 104 or standalone sensor 108 ofrobot 100, so that the user can perform debugging analysis.

One or more remote computing systems 160 may be in communication withrobot over different communication channels, such as wirelesscommunication channels 120A and/or 120B. Remote computing system 160 maycome in various forms and may be used for various purposes pertaining tooperation of robot 100. For example, remote computing system 160 couldsimply be a debugging workstation with components similar to those ofdebugging workstation 150, except that remote computing system 160 isonly in communication with robot 100 over wireless channels 120A and/or120B, rather than wired communication channel 118. As another example,remote computing system 160 could be a computing device operating asoftware application that may be used to exert various degrees ofcontrol over robot 100, e.g., a program operable to “check out” robot100. The more autonomy robot 100 has, the less control such anapplication may exert over robot.

FIG. 2 schematically depicts in more detail than FIG. 1 an example ofhow robot 100 may offload and/or store sensor data pursuant to one ormore transport rules, in accordance with various implementations. Robot100 operates, e.g., in memory 110, an instance of a sensor datamanagement engine 130. Sensor data management engine 130 may beconfigured to obtain sensor data sensed by one or more sensors, such asstandalone sensors 108 and/or sensors associated with operationalcomponents 104, tag that data for storage in one or more buffers 270 ofmemory 110 (e.g., a ring buffer), and treat that data pursuant to one ormore transport rules, e.g., by offloading at least a subset of thesensor data through one or more communication channels (e.g., 118,120A-B), by selectively storing a subset of the sensor data, and/or bydisregarding some of the sensor data. In this example, sensor datamanagement engine 130 includes a sensor data tagger 272, a contextengine 274, a transport rule engine 276, and a sensor data distributor278.

Sensor data tagger 272 may be configured to identify various groups ofsensor data points that are sampled by one or more sensors 108/104associated with robot 100 and that share an attribute, and to tag theidentified data points with a tag that is indicative of the attributeshared by the data points. How sensor data tagger 272 tags sampledsensor data points may be determined in various ways. In someimplementations, a user may configure robot 100, e.g., directly and/orthrough a user interface (e.g., provided by user interface engine 152),to tag particular sensor data in various ways. In some implementations,robot 100 may be manufactured with a default sensor data point taggingscheme that, for instance, tags each sensor data point with a tagindicative of its origin.

In various implementations, attributes that may be used as bases fortagging sensor data points may come in various forms, such as an originof a sensor data point (which sensor/operational component sampled it?),a data type of a data point (e.g., float, double, torque measurement,temperature, etc.), and/or a context in which the data point was sampled(e.g., during normal operation, debugging, error response mode, robotcalibration, etc.). In some implementations, a sensor data point may betagged with a single tag. In some implementations, a sensor data pointmay be tagged with multiple data tags. To “tag” a sensor data point maymean to store, e.g., in a memory location of buffer 270 that isassociable with another memory location of buffer 270 that stores orwill store the sensor data point itself, data that is indicative of theattribute shared by the sensor data point with other sensor data points.For example, in some implementations, a sensor data point and itsassociated tag(s) may be stored in contiguous memory locations of buffer270. In other implementations, a sensor data point may be stored with apointer that points to another memory location storing a tag associatedwith the sensor data point.

As noted in the background, no matter how much storage capacity memory110 has, sensors such as standalone sensors 108 and/or operationalcomponents 104 may produce so much data that memory 110 can easilybecome full. And in some instances, all or a portion of memory 110 maybe implemented as a ring or circular buffer that is treated as though itis connected end-to-end, such that data written to the ring buffer isperiodically overwritten as a matter of course. Accordingly, variousother components of sensor data management engine 130 may be configuredto treat tagged sensor data points in various ways (e.g., offloading,storing, disregarding/overwriting) so that important sensor data is lesslikely to be overwritten.

Context engine 274 may be configured to determine, at any given point intime, a context in which robot 100 operates. The term “context” as usedherein may refer to the circumstances in which robot 100 operates, suchas particular communication channels available or not available to robot100, an activity level of robot 100, an amount of charge available torobot 100 (e.g., how much power is left in its battery), an amount ofmemory available within robot 100, and so forth. Context engine 274 mayprovide data indicative of a current context of robot 100 to transportrule engine 276.

Transport rule engine 276 may be configured to identify one or moretransport rules stored in an index 277. As noted above, a transport rulemay govern how data points tagged with a particular tag are treated whenrobot 100 operates in a particular context. In some implementations,each possible context of the robot may be associated with a plurality oftransport rules, and each transport rule may define how data tagged witha particular tag should be treated when the robot operates in thatcontext. In other implementations, each sensor data tag may beassociated with a plurality of transport rules, and each transport rulemay define how data tagged with the sensor data tag should be treated ina particular context. In various implementations, index 277 may includea lookup table or similar mechanism that may be consulted, e.g., bytransport rule engine 276, to determine which transport rules areapplicable to sensor data tagged with a particular tag or tags.Transport rule engine 276 may provide transport rules it identifies tosensor data distributor 278.

Sensor data distributor 278 may be configured to selectively offload orstore at least a subset of various groups of commonly-tagged sensor datapoints pursuant to one or more transport rules received from transportrule engine 276 points while robot 100 operates in the contextidentified by context engine 274. For example, in FIG. 2, sensor datadistributor 278 may provide all or a subset of sensor data points taggedwith a first tag to first antenna 116A, all or a subset of sensor datapoints tagged with a second tag to second antenna 116B, and mayselectively store all or a subset of sensor data points tagged with athird tag in buffer 270. In some implementations, sensor data pointsstored in buffer 270 by sensor data distributor 278 may be stored theretemporarily (and in some instances may be locked to prevent overwriting)until robot 100 has a communication channel available that is moresuitable to offload this sensor data.

Transport rules may govern how tagged data is offloaded from robot 100or stored in innumerable ways. For example, one transport rule mayprovide that highly critical sensor data tagged with a first tag isoffloaded via a first, relatively high bandwidth channel (e.g., Wi-Fi).The transport rule may likewise provide that less critical sensor datatagged with a second tag is offloaded via another channel (e.g.,Bluetooth). In some instances, sensor data tagged with a particular tagmay be offloaded in an amount and/or manner that is commensurate with areliability or available communication bandwidth of a channel. Forexample, if a high bandwidth, high reliability channel is available, atransport rule may provide that all sensor data tagged with a particulartag may be offloaded through the channel. However, if a lower bandwidth,lower reliability channel is all that is available, the same transportrule or a different transport rule may provide that a compressed subsetof the tagged sensor data is offloaded, or that only every xth sample isoffloaded, or that a compressed subset is stored in buffer 270 of robot100 until a scheduled rendezvous between robot 100 and a chargingstation (during which robot 100 may have a high bandwidth dataconnection with a remote computing device), etc. These are just a fewexamples.

FIG. 3 depicts an example “SELECT SENSOR DATA TO RETRIEVE” graphicaluser interface 380 that may be provided, e.g., by user interface engine152 of debugging workstation 150, in accordance with variousimplementations. A robot 300 is rendered (in this example, robot 300takes the form of a robot arm) in a multidimensional representation 382of a robot environment. Four sensor data sources 384-D are visuallyemphasized. First sensor data source 384A is an end effector of therobot arm; the remaining sensor data sources 384B-D are from operationalcomponents in the form of joints. In some implementations, sensorsources may be automatically visually emphasized, e.g., by userinterface engine 152, based on different tags used to tag sensor data.In this example, there appear to have been four different tags used totag sensor data produced by robot 300, and so there are four selectablesensor sources 384. However, if more or less different types of tags areused to tag sensor data, then more or less respective selectable sensorsources 384 may be rendered.

A user may select one or more sensor data sources 384, e.g., with amouse or on a touch screen with the user's finger, to toggle betweenstates in which all, various subsets of, or no sensor data produced bythese sources is provided, e.g., to debugging workstation 150 forstorage by logging engine 154 in index 156. Additional user interfaceelements 386 may be provided that enable a user to select contexts inwhich robot 300 can operate, including normal operation, calibration,charging, and error response and to configure how sensor data isoffloaded/stored in those contexts. For instance, by selecting one ormore of these graphical elements 386, a user can toggle whether shewishes to receive or not receive sensor data while robot 300 operatescorresponding contexts.

Referring now to FIG. 4, an example method 400 of tagging sensor dataand treating tagged sensor data pursuant to transport rules is depicted.For convenience, the operations of flow charts are described withreference to a system that performs the operations. This system mayinclude various components of various computer systems, includingelements of robot 100 and/or debugging workstation 150. Moreover, whileoperations of method 400 are shown in a particular order, this is notmeant to be limiting. One or more operations may be reordered, omitted,or added. At block 402, the system may identify a first plurality ofsensor data points that are sampled by one or more sensors associatedwith a robot and that share a first attribute. For example, sensor datatagger 272 of robot 100 may identify data points that were sampled by atorque sensor associated with a particular operational component 104.

At block 404, the system may tag each of the sensor data pointsidentified at block 402 with a first tag that is indicative of the firstattribute. For example, if each memory location that is to store anidentified sensor data point will have sufficient bits to spare, thesystem may store a tag in the spare bits. If each memory location willnot have sufficient bits to spare, then an adjacent memory location, ora memory location that is readily identified and accessible, may be usedinstead to store the tag. Sensor data may be tagged in various ways thatmay be selected by a user (e.g., a robot technician) and/or may occurautomatically. For example, some sensors, such as thermometers, may besampled at a relatively high frequency to produce a lot of raw data, butin many scenarios, only a subset of data points produced by such asensor may be of interest. Accordingly, a user may cause such datapoints to be tagged and then assign transport rules to the tag thatensure that only a subset of the raw data is ever offloaded or stored,no matter what the context. Should the user later wish to see all of theraw data, the user may, for instance, operate debugging workstation tocause all sensor data tagged with that tag to be offloaded from robot100.

At block 406, the system may, e.g., by way of context engine 274,identify a context in which robot 100 is operating. For example, robot100 (and in particular, logic 102) may scan one or more wiredcommunication interfaces 112 and/or one or more wireless communicationinterfaces 114 to determine what communication channels are currentlyavailable to robot 100. Additionally or alternatively, logic 102 maydetermine how much storage remains in memory 110, e.g., in buffer 270.In some implementations, logic 102 may be more prone to attempt tooffload sensor data if storage is low.

Additionally or alternatively, logic 102 may, as part of identifying acontext of robot 100, determine an activity level of robot 100. If robot100 is very active, e.g., moving quickly around an environment such thatstrength and/or reliability of one or more wireless communicationchannels available to robot 100 is likely to fluctuate, then robot 100may be more prone to cache sensor data locally, at least until robot 100is less active (e.g., when robot 100 visits a charging station).Additionally or alternatively, logic 102 may determine how much chargeremains in a battery (not depicted) of robot 100. If robot 100 is low oncharge, and especially if robot will be visiting a charging station soonwhere it will have stronger communication channels available to it,robot 100 may be less prone to offload sensor data.

At block 408, the system may identify a first transport rule thatgoverns how data points tagged with the first tag are treated when therobot operates in the context identified at block 406. In someimplementations, multiple contextual signals may be considered and/orweighed by logic 102 when identifying, selecting, and/or applyingtransport rules, e.g., to select the “best” transport rule and/or toresolve conflicts. Suppose Transport Rule A is applicable in contexts inwhich robot 100 is low on power, and provides that robot 100 shouldstore sensor data tagged “XYZ” locally until robot 100 returns to acharging station. Suppose further that Transport Rule B is applicable insituations in which robot 100 has little to no storage capacityremaining in memory 110, and provides that robot 100 should be moreaggressive than usual to offload sensor data tagged “XYZ” to avoid thesensor data being overwritten (e.g., in a ring buffer). Now, supposerobot 100 is operating with low power, but that it also is low onstorage in memory 110. While both Transport Rules A and B areapplicable, they are also in conflict.

Various techniques may be employed to resolve such conflicts. Forexample, in some implementations, Transport Rule A would have a higher“priority” than Transport Rule B, e.g., because an operator would ratherlose sensor data than have robot 100 run out of power before it reachesa charging station. In some implementations, other contextual signalsand/or transport rules may be considered to break the tie. Suppose that,in the scenario described above, robot 100 currently has a wirelesscommunication channel available to it that is relatively low bandwidth,but has at least acceptable reliability. A third transport rule, e.g.,Transport Rule C, may provide that logic 102 of robot 100 cull and/orotherwise compress sensor data tagged “XYZ” for offloading over such awireless channel when robot 100 is low on power and/or available memory.That way, at least some of the sensor data tagged “XYZ” is preserved,and robot 100 may be prevented from losing power before it can reach acharging station.

Referring back to FIG. 4, at block 410, the system may offload or storeat least a subset of the first plurality of tagged sensor data pointspursuant to the one or more transport rules identified at block 408. Asnoted above, in some implementations, logic 102 may select a subset ofthe tagged sensor data, e.g., by compressing the raw data, subsamplingthe data (e.g., selecting every xth data point), and so forth, and mayonly offload and/or store that subset of data. In contexts in whichcommunication channels available to robot 100 are limited (but stillavailable to at least some extent), this may constrain the amount ofdata robot 100 attempts to offload. In contexts in which communicationchannels are not available or insufficient, this may enable robot 100 tostore at least some tagged sensor data locally without taking up aninordinate amount of memory.

FIG. 5 is a block diagram of an example computer system 510. Computersystem 510 typically includes at least one processor 514 whichcommunicates with a number of peripheral devices via bus subsystem 512.These peripheral devices may include a storage subsystem 524, including,for example, a memory subsystem 525 and a file storage subsystem 526,user interface output devices 520, user interface input devices 522, anda network interface subsystem 516. The input and output devices allowuser interaction with computer system 410. Network interface subsystem516 provides an interface to outside networks and is coupled tocorresponding interface devices in other computer systems.

User interface input devices 522 may include a keyboard, pointingdevices such as a mouse, trackball, touchpad, or graphics tablet, ascanner, a touchscreen incorporated into the display, audio inputdevices such as voice recognition systems, microphones, and/or othertypes of input devices. In general, use of the term “input device” isintended to include all possible types of devices and ways to inputinformation into computer system 510 or onto a communication network.

User interface output devices 520 may include a display subsystem, aprinter, a fax machine, or non-visual displays such as audio outputdevices. The display subsystem may include a cathode ray tube (CRT), aflat-panel device such as a liquid crystal display (LCD), a projectiondevice, or some other mechanism for creating a visible image. Thedisplay subsystem may also provide non-visual display such as via audiooutput devices. In general, use of the term “output device” is intendedto include all possible types of devices and ways to output informationfrom computer system 510 to the user or to another machine or computersystem.

Storage subsystem 524 stores programming and data constructs thatprovide the functionality of some or all of the modules describedherein. For example, the storage subsystem 524 may include the logic toperform selected aspects of method 400, and/or to implement one or moreaspects of robot 100 and/or debugging workstation 150. Memory 525 usedin the storage subsystem 524 can include a number of memories includinga main random access memory (RAM) 530 for storage of instructions anddata during program execution and a read only memory (ROM) 532 in whichfixed instructions are stored. A file storage subsystem 526 can providepersistent storage for program and data files, and may include a harddisk drive, a CD-ROM drive, an optical drive, or removable mediacartridges. Modules implementing the functionality of certainimplementations may be stored by file storage subsystem 526 in thestorage subsystem 524, or in other machines accessible by theprocessor(s) 514.

Bus subsystem 512 provides a mechanism for letting the variouscomponents and subsystems of computer system 510 communicate with eachother as intended. Although bus subsystem 512 is shown schematically asa single bus, alternative implementations of the bus subsystem may usemultiple busses.

Computer system 510 can be of varying types including a workstation,server, computing cluster, blade server, server farm, smart phone, smartwatch, smart glasses, set top box, tablet computer, laptop, or any otherdata processing system or computing device. Due to the ever-changingnature of computers and networks, the description of computer system 510depicted in FIG. 5 is intended only as a specific example for purposesof illustrating some implementations. Many other configurations ofcomputer system 510 are possible having more or fewer components thanthe computer system depicted in FIG. 5.

While several implementations have been described and illustratedherein, a variety of other means and/or structures for performing thefunction and/or obtaining the results and/or one or more of theadvantages described herein may be utilized, and each of such variationsand/or modifications is deemed to be within the scope of theimplementations described herein. More generally, all parameters,dimensions, materials, and configurations described herein are meant tobe exemplary and that the actual parameters, dimensions, materials,and/or configurations will depend upon the specific application orapplications for which the teachings is/are used. Those skilled in theart will recognize, or be able to ascertain using no more than routineexperimentation, many equivalents to the specific implementationsdescribed herein. It is, therefore, to be understood that the foregoingimplementations are presented by way of example only and that, withinthe scope of the appended claims and equivalents thereto,implementations may be practiced otherwise than as specificallydescribed and claimed. Implementations of the present disclosure aredirected to each individual feature, system, article, material, kit,and/or method described herein. In addition, any combination of two ormore such features, systems, articles, materials, kits, and/or methods,if such features, systems, articles, materials, kits, and/or methods arenot mutually inconsistent, is included within the scope of the presentdisclosure.

What is claimed is:
 1. A method comprising: identifying, by a robot,first and second pluralities of sensor data points that are generatedduring operation of the robot while the robot performs one or moretasks, wherein the first plurality of sensor data points share a firstattribute and the second plurality of sensor data points share a secondattribute that is different than the first attribute, and wherein thesensor data points are generated by one or more cameras integral withthe robot or one or more light sensors integral with the robot; tagging,by the robot, each of the identified first plurality of sensor datapoints with a first tag, wherein the first tag is indicative of thefirst attribute shared by the first plurality of sensor data points;tagging, by the robot, each of the identified second plurality of sensordata points with a second tag, wherein the second tag is indicative ofthe second attribute shared by the second plurality of sensor datapoints, and wherein the second tag also indicates a lower priority thanthe first tag; identifying, by the robot, a context in which the robotis operating; identifying, by the robot, a first transport rule thatgoverns how sensor data points tagged with the first tag are treatedwhen the robot operates in the context; identifying, by the robot, asecond transport rule that governs how sensor data points tagged withthe second tag are treated when the robot operates in the context;offloading, by the robot to a remote computing device via a networkconnection, at least a subset of the first plurality of tagged sensordata points pursuant to the first transport rule, wherein the offloadingcomprises transmitting at least the subset of the first plurality oftagged sensor data points to the remote computing device in a mannerselected based on the context in which the robot is operating; andoverwriting, by the robot, at least a subset of the second plurality oftagged sensor data points pursuant to the second transport rule.
 2. Themethod of claim 1, wherein the subset is selected from the firstplurality of tagged sensor data points based on the context in which therobot is operating.
 3. The method of claim 1, wherein the networkconnection is selected based on the context.
 4. The method of claim 1,wherein the context comprises available communication bandwidth betweenthe robot and the remote computing device.
 5. The method of claim 1,wherein the context comprises reliability of the network connection. 6.The method of claim 1, wherein the context comprises power available tothe robot.
 7. The method of claim 1, wherein the context comprisesavailable memory in the robot.
 8. The method of claim 1, wherein thecontext comprises an activity level of the robot.
 9. The method of claim1, wherein the one or more vision sensors include a proximity sensor.10. A method comprising: identifying, by a robot, first and secondpluralities of sensor data points that are generated during operation ofthe robot while the robot performs one or more tasks, wherein the firstplurality of sensor data points share a first attribute and the secondplurality of sensor data points share a second attribute that isdifferent than the first attribute, and wherein the sensor data pointsare generated by one or more cameras integral with the robot or one ormore pressure wave sensors integral with the robot; tagging, by therobot, each of the identified first plurality of sensor data points witha first tag, wherein the first tag is indicative of the first attributeshared by the first plurality of sensor data points; tagging, by therobot, each of the identified second plurality of sensor data pointswith a second tag, wherein the second tag is indicative of the secondattribute shared by the second plurality of sensor data points, andwherein the second tag also indicates a lower priority than the firsttag; identifying, by the robot, a context in which the robot isoperating; identifying, by the robot, a first transport rule thatgoverns how sensor data points tagged with the first tag are treatedwhen the robot operates in the context; identifying, by the robot, asecond transport rule that governs how sensor data points tagged withthe second tag are treated when the robot operates in the context;offloading, by the robot to a remote computing device via a networkconnection, at least a subset of the first plurality of tagged sensordata points pursuant to the first transport rule, wherein the offloadingcomprises transmitting at least the subset of the first plurality oftagged sensor data points to the remote computing device in a mannerselected based on the context in which the robot is operating; andoverwriting, by the robot, at least a subset of the second plurality oftagged sensor data points pursuant to the second transport rule.
 11. Themethod of claim 10, wherein the subset is selected from the firstplurality of tagged sensor data points based on the context in which therobot is operating.
 12. The method of claim 10, wherein the networkconnection is selected based on the context.
 13. The method of claim 10,wherein the context comprises available communication bandwidth betweenthe robot and the remote computing device.
 14. The method of claim 10,wherein the context comprises reliability of the network connection. 15.The method of claim 10, wherein the context comprises power available tothe robot.
 16. The method of claim 10, wherein the context comprisesavailable memory in the robot.
 17. The method of claim 10, wherein thecontext comprises an activity level of the robot.
 18. A methodcomprising: identifying, by a robot, first and second pluralities ofsensor data points that are generated during operation of the robotwhile the robot performs one or more tasks, wherein the first pluralityof sensor data points share a first attribute and the second pluralityof sensor data points share a second attribute that is different thanthe first attribute, and wherein the sensor data points are generated bytwo or more of an accelerometer integral with the robot, a gyroscopeintegral with the robot, a force sensor integral with the robot, and atorque sensor integral with the robot; tagging, by the robot, each ofthe identified first plurality of sensor data points with a first tag,wherein the first tag is indicative of the first attribute shared by thefirst plurality of sensor data points; tagging, by the robot, each ofthe identified second plurality of sensor data points with a second tag,wherein the second tag is indicative of the second attribute shared bythe second plurality of sensor data points, and wherein the second tagalso indicates a lower priority than the first tag; identifying, by therobot, a context in which the robot is operating; identifying, by therobot, a first transport rule that governs how sensor data points taggedwith the first tag are treated when the robot operates in the context;identifying, by the robot, a second transport rule that governs howsensor data points tagged with the second tag are treated when the robotoperates in the context; offloading, by the robot to a remote computingdevice via a network connection, at least a subset of the firstplurality of tagged sensor data points pursuant to the first transportrule, wherein the offloading comprises transmitting at least the subsetof the first plurality of tagged sensor data points to the remotecomputing device in a manner selected based on the context in which therobot is operating; and overwriting, by the robot, at least a subset ofthe second plurality of tagged sensor data points pursuant to the secondtransport rule.